Many different types of so-called "special-effects" can be created using digital imaging techniques. One such class of special-effects techniques involves inserting the foreground of one image into a different background image. This makes it possible for a person or object to appear to be in a different setting than they really are. For example, the weatherman can appear to be standing in front of a weather map, when in reality he is standing in front of a blue wall, or an actor can appear to be standing on the edge of a cliff, when in reality he is actually performing in the safety of a studio. Typically, these methods rely on having the foreground object photographed in front of a brightly colored backdrop of a known color. A common backdrop color is blue, which is why this technique is often referred to as "blue-screening."
The basic steps involved with implementing a typical blue-screening algorithm are illustrated in FIG. 1. First, an object is photographed in front of a brightly colored backdrop of a known color which is shown as an image capture step 10. The captured image will contain a foreground region corresponding to the object being photographed, and a key color region, corresponding to the brightly colored backdrop. The key color region has a predetermined key color such as bright green or bright blue.
A segmentation step 12 is next used to segment the captured image into the foreground region and the key color region by detecting portions of the image having the key color. Since the color of the backdrop will not be perfectly constant, the key color will typically be characterized by a range of color values surrounding some nominal color value, rather than a single point in color space.
Many blue-screening algorithms also include a transition signal creation step 14. This is useful because the image will typically contain some foreground pixels that have been contaminated by the key color. For example, the pixels that occur along the boundary between the foreground region and the key color region usually contain a mixture of foreground color and key color. The transition signal is determined to indicate the relative amount of foreground color and key color contained in each contaminated pixel.
Finally, an image composition step 16 is used to combine the foreground region of the captured image with a second background image. During this step, the foreground region of the captured image is inserted into the background image. For the foreground pixels that were determined to be contaminated with the key color, the transition signal can be used to remove the appropriate amount of the key color and replace it with the corresponding background image.
Several methods have been disclosed in the prior art for the image segmentation step 12 shown in FIG. 1. These methods generally involve converting the image data into a luminance-chrominance color space representation such as the well-known YCrCb or CIELAB color spaces. A key color zone is then defined relative to the chrominance coordinates so that image colors that fall within this key color zone will be classified as belonging to the key color region. Examples of typical key color zones that can be found in the prior art are shown in FIG. 2. (These examples were taken from U.S. Pat. Nos. 4,533,937, 5,381,184 and 5,455,633.) In each of these examples, the key color zone is indicated by a cross-hatched area in a Cr-Cb chrominance plane. FIG. 2(a) shows a half-plane key color zone 20, FIG. 2(b) shows a circular key color zone 22, FIG. 2(c) shows a wedge shaped key color zone 24, and FIG. 2(d) shows a diamond key color zone 26.
The prior art key color zones shown in FIG. 2 are all defined relative to the chrominance coordinates, and are therefore independent of the Y luminance value, so that color values in the image at all brightness levels are segmented using the same zone boundary. In cases where the key color backdrop is illuminated uniformly, the key color zone can usually be identified quite accurately. This is frequently done by capturing an image of the key color backdrop with no foreground objects in place. However, when objects are actually placed in front of the backdrop, the effective color of the key color backdrop will change somewhat. This is due to the presence of shadows cast onto the key color backdrop by the subject, or other causes such as secondary illumination of the backdrop by light reflected off the foreground objects. As a result, the key color zone typically needs to be large enough to include both the shadowed and unshadowed regions of the key color backdrop.
Actual object colors that fall within the key color zone will also be classified as belonging to the chroma key region. This is sometimes referred to as a "false positive" error. When a false positive error occurs, some region of the foreground object will be classified as key color, and will therefore be replaced with the background image during the image composition step 16. For example, if the weatherman were wearing a tie that was the same color as the key color backdrop, the weather map in the background would appear to show through a tie-shaped hole in his chest.
For television and movie studio applications, these false positive errors are usually avoided by carefully controlling the colors that occur in the foreground objects. Therefore, the weatherman would never be permitted to wear a tie that matched the color of the key color backdrop. However, in other applications it is not possible to have tight control over the foreground colors. For example, if a kiosk were set up in an amusement park to produce composite pictures of customers posed in front of a variety of backgrounds, the system operator will not have any control over the color of the clothing that the customers will be wearing. As a result, the probability of false positive errors occurring increases substantially. Therefore, in these cases, it is critical that the key color zone be made as small as possible. However, as explained earlier, the prior art implementations require a large key color zone to handle conditions such as shadows. Additionally, the fact that the chroma key zones are defined relative to the two-dimensional chrominance coordinates makes it impossible to account for the fact that the key color will actually be characterized by a three-dimensional volume in color space.